Bias-Variance Tradeoff

Bias-Variance trade-off -> Underfitting vs. Overfitting issues

Bias & Variance in ML

Evaluate bias & variance

You can evaluate the bias and variance by checking errors of the train and dev sets.

Train set error Dev set error type
low (1%) high (11%) high variance
high (15%) high, but near train set error (16%) high bias
high (15%) high, but much higher than Train set error (30%) high bias & high variance
low (0.5%) low (1%) low bias & low variance

Regularization influences bias-variance trade-off

check Cost Functions#Cost function with regularization and Regularization

bias-variance-1.png|600

bias-variance-4.png|600

learning curves

bias-variance-2.png|300
bias-variance-3.png|300
increasing training set size can...

How to fix

Bias & Variance in Deep Learning

Different from simple ML models, there is rarely trade-off between bias-variance in DL models. You can now reduce bias without hurting variance, and vice versa.

Pasted image 20230313213430.png|400